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Gun Shows and Gun Violence: Fatally Flawed Study Yields Misleading Results 


| Garen J. Wintemute, MD, MPH, David Hemenway, PhD, Daniel Webster, ScD, MPH, Glenn Pierce, PhD, and Anthony A. Braga, PhD 


A widely publicized but un- 
published study of the rela- 
tionship between gun shows 
and gun violence is being 
cited in debates about the reg- 
ulation of gun shows and gun 
commerce. We believe the 
study is fatally flawed. 

A working paper entitled 
“The Effect of Gun Shows on 
Gun-Related Deaths: Evidence 


from California and Texas” 
outlined this study, which 
found no association between 
gun shows and gun-related 
deaths. We believe the study 
reflects a limited understand- 
ing of gun shows and gun 
markets and is not statistically 
powered to detect even an 
implausibly large effect of 
gun shows on gun violence. 
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In addition, the research con- 
tains serious ascertainment 
and classification errors, pro- 
duces results that are sensi- 
tive to minor specification 
changes in key variables and 
in some cases have no face 
validity, and is contradicted 
by 1 of its own authors’ prior 
research. The study should 
not be used as_ evidence 


in formulating gun_ policy. 
(Am J Public Health. 2010; 
100:1856-1860. doi:10.2105/ 
AJPH.2010.191916) 


IN EARLY OCTOBER 2008, THE 
National Bureau of Economic 
Research posted on its Web site 
a working paper by Duggan et al. 
titled “The Effect of Gun Shows on 
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Gun-Related Deaths: Evidence 
from California and Texas.”' A 
University of Michigan press release 
announced the study with the 
headline “Gun shows do not in- 
crease homicides or suicides” and 
began as follows: 


A new study finds no evidence 
that gun shows lead to substantial 
increases in either gun-related 
homicides or suicides. 

The. . study also shows that 
tighter regulation of gun shows 
does not appear to reduce the 
number of firearms-related 
deaths.” 

Not surprisingly, the paper re- 
ceived nationwide publicity. Just 
6 weeks after its release, it was 
the topic of a relentlessly favor- 
able cover article in America’s 1° 
Freedom, the monthly official 
journal of the National Rifle As- 
sociation.® 

To our knowledge, the working 
paper has not been published in 
a peer-reviewed publication. An- 
ecdotal reports from policy advo- 
cates and policymakers suggest 
that it has nonetheless become 
influential in the continuing de- 
bate about how best to regulate 
gun shows and gun commerce to 
prevent violence. We believe that 
the study discussed in the working 
paper contains serious errors in 
design and execution that fatally 
compromise its findings. Corre- 
spondence with the authors*° 
did not lessen our concerns and, 
because the paper remains in 
circulation, we seek to share those 
concerns with the research and 
policy communities. For brevity’s 
sake we confine our comments 
largely to the study's findings for 
homicide, although we have similar 
concerns about the study’s findings 
for suicide. 
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LIMITS TO THE STUDY 
DESIGN 


Large gun shows can usefully 
be thought of as the big box re- 
tailers of gun commerce.” Dozens 
to hundreds of gun sellers, both 
licensed retailers and private 
parties, are present and compete for 
the business of thousands of cus- 
tomers. Surveys suggest that from 
4% to 9% of gun sales nationwide 
occur at gun shows.® Most sales at 
gun shows are made by licensed 
retailers, and most private-party 
gun sales are made elsewhere.°” 
Federal law makes no distinction 
between sales occurring at gun 
shows and other sales; there is no 
such thing as a gun show loophole. 
Sales by licensed retailers require 
a background check, substantial 
paperwork, and a permanent re- 
cord, but sales by private parties 
can be completely anonymous and 
undocumented.° Seventeen states 
require background checks for pri- 
vate-party handgun sales at gun 
shows, and 10 do so for gun show 
sales of rifles and shotguns.’° (Only 
6 states require background checks 
for all private-party gun sales."°) 

The relationship between gun 
shows and gun-related crime is 
complex. Although they are 
a source of the guns used in crime, 
gun shows are just 1 of the venues 
that supply firearms for criminal 
use on a more-or-less continuing 
basis. Surveys of persons incar- 
cerated for gun crimes suggest that 
less than 2% of them acquired 
their weapons at gun shows." 
Gun shows are more frequently an 
intermediate source of crime guns, 
supplying organized gun trafficking 
operations and surrogate or “straw” 
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purchasers who in turn supply 
criminal end users.°’"3"* A review 
of federal investigations initiated 
between July 1996 and December 
1998 of illegal gun trafficking 
found that nearly 26 000 trafficked 
guns—30% of all guns linked to the 
investigations—had been sold at 
gun shows.”° 

Duggan et al. hypothesized that 
gun shows have short-term and 
local effects on homicide (and sui- 
cide). Their data are for California 
and Texas from 1994 to 2004. To 
test their hypothesis, they deter- 
mined whether there was any as- 
sociation between the number of 
deaths in a given location (specified 
as a zip code) in a given week and 
the number of gun shows in or near 
that location in the preceding 4 
weeks. Their unit of analysis is the 
zip codex week, and their models 
variably examined zip codes for 
areas in which gun shows occur and 
codes for areas that were within 
5, 10, and 25 miles of gun shows. 
Duggan et al. acknowledged these 
time and space restrictions as a lim- 
itation,' but in our judgment the 
restrictions lead to fundamental 
questions about the study’s design 
and the validity of its findings. 

Many studies have established 
that gun crimes rarely involve 
guns purchased within a few 
weeks of the crime’s occur- 
rence.°"® Those studies relied on 
data from gun tracing, which is 


[T]he systematic process of 
tracking a recovered crime gun’s 
history from its source (manu- 
facturer/importer) through the 
chain of distribution (wholesaler/ 
retailer) to the individual who 
first purchases the firearm!°?\ 


In preparing this critique we 
have analyzed tracing data 


compiled by the Bureau of Alco- 
hol, Tobacco, Firearms and Ex- 
plosives for guns recovered in 
California and Texas from 2003 
to 2006. Only 1.3% of traced 
crime guns from those states were 
recovered by law enforcement 
agencies within 4 weeks of their 
first retail sale. 

Similarly, Duggan et al. restrict 
their analysis to events occurring 
within at most 25 miles of a 
gun show, which ignores well- 
established findings about the ge- 
ography of illegal gun markets. 
As a general rule, as many as 
one-third of crime guns recovered 
in most cities are purchased out 
of state, and another third are 
purchased in the same state but 
outside the immediate vicinity.’° 
Our analysis of data for traced guns 
recovered in California and Texas 
from 2003 to 2006 finds that 
49.5% were recovered within 25 
miles of their point of initial sale. 
Additional data from that study, 
which is still in progress, suggest 
that guns purchased at gun shows 
travel an even greater distance from 
their point of sale to their place of 
use in crime than do guns pur- 
chased in other venues. (G.P., A.B., 
and G.W., unpublished manuscript, 
2010). 

The combined impact of the 
temporal and geographic con- 
straints imposed by Duggan et al. 
is demonstrated in Table 1. From 
2003 to 2006, just 1.0% of guns 
involved in crimes in Texas and 
California were recovered within 
28 days and 25 miles of their 
entry into the market. 

It is important to note as a 
limitation to our analysis that 
standard gun traces terminate 
with the first retail sale of a gun, 
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and both new and used guns are 
sold at gun shows. But when gun 
traces are extended to the most 
recent recorded sale—as is possi- 
ble for handguns in California, for 
which a centralized archive of 
sales data exists and where pri- 
vate-party sales must be routed 
through licensed retailers—overall 
results are similar to those 
obtained using standard gun 
traces.'® 

It is entirely possible that gun 
shows have long-term and far 
reaching effects on homicide (and 
suicide), without exhibiting the 
short-term, local effects hypothe- 
sized by Duggan et al. In most 
urban areas gun shows occur less 
than once a month.° Potential 
offenders who want a gun imme- 
diately are unlikely to wait for 
a gun show. As discussed, gun 
shows may be a more likely 
choice for criminal gun trafficking 
operations, which often move 
guns long distances across state 
lines or our borders with Mexico 
and Canada.°"3 


LACK OF STATISTICAL 
POWER 


The time and distance con- 
straints imposed by Duggan et al. 
create an insurmountable statisti- 
cal power problem. Given those 
constraints and the data in Table 1, 
it is reasonable to infer that their 
analysis misses the great major- 
ity—as much as 99%-—of the 
expected effect of gun shows on 
homicide. The following highly 
favorable power calculation finds 
that the analyses by Duggan et al. 
have less than 10% power for 
finding even a very large effect of 
gun shows on homicide. 
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TABLE 1—Time and Distance From First Sale by a Licensed Retailer to Recovery by a Law Enforcement 
Agency for Guns in California and Texas: 2003-2006 


Miles from Purchase to Recovery 


Time from Gun Sale to Recovery by Police 0-25, No. (%) 226, No. (%) Total, No. (%) 
1-28 d 680 (1.0) 284 (0.4) 964 (1.4) 
29-90 d 1009 (1.4) 442 (0.6) 1451 (2.1) 
91-365 d 2710 (3.9) 1699 (2.4) 4409 (6.3) 
2366 d 30260 (43.2) 32 894 (47.0) 63154 (90.2) 
Total 34659 (49.5) 35319 (50.5) 69978 (100.0) 


Note. Percentages are of all guns in the table. Data were missing for another 79 122 guns. For a gun to be added to the table, dates and places 
for both sale and recovery were required. Data were missing for a variety of reasons: a retailer may not be able to provide a date of sale, for 
example, or the serial number on the gun may have been obliterated, making tracing impossible. Some traces for handguns recovered in 

California may have been extended beyond the first retail sale, as the state maintains a computerized archive of handgun purchases that is used 
for crime gun tracing. With few exceptions, traced guns have been used in a crime or recovered under circumstances that suggest criminal use. 


For the power calculation, let us 
err in favor of the original study 
and assume that (1) 5% of homi- 
cides committed with guns pur- 
chased at gun shows occur within 
4 weeks and 25 miles of those 
shows, and (2) the total effect of 
gun shows is to increase gun ho- 
micide by an implausibly large 
30%, regardless of time and place. 
This means the study is attempting 
to identify a 1.5% increase in 
homicides (5% 30%). Let us also 
assume that the control group size 
is 900000 (the maximum con- 
trol group size for California), and 
the size of the intervention group 
(gun show counties) is 20000. 

Using findings from Duggan 
et al.' of a mean gun homicide rate 
per county of 0.035 deaths per 
100000 persons per week with 
a standard deviation of 0.203, then 
for a 2-sample test, 1-tailed, with an 
a of 5%, the power is 10%. 

In other words, Duggan et al. 
had only a 10% chance of detect- 
ing even an implausibly large ef- 
fect of gun shows on homicide. 
Given this result, the fact that the 
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Source. G. Pierce, A. Braga, and G. Wintemute, unpublished manuscript, 2010. 


they did not find an effect of gun 
shows on homicide cannot 

be taken to mean that no such 
effect exists. As we have suggested, 
that effect may simply be exerted 
over a longer time period and 
broader geographic area. 


DATA DEFICIENCIES 


Duggan et al. used 1 publication, 
the Gun & Knife Show Calendar, to 
identify gun shows. But prior re- 
search®"* shows that no single 
source is adequate for this purpose. 
Another widely available listing 
is in the Big Show Journal. To de- 
termine the magnitude of misclas- 
sification that might exist in data 
obtained from a single source, we 
compiled data from both publica- 
tions for 2007 and found that 
298 gun shows were held in Cal- 
ifornia or Texas that year. Only 236 
(79%) of these were listed in the 
Gun & Knife Show Calendar. This 
suggests that Duggan et al. failed 
to identify roughly 20% of the 
gun shows occurring in those states 
during their study period—some 


680 events altogether. Based on the 
listings for 2007, some of these 
missed shows would have been 
large events with many gun sales. 

If, as Duggan et al. hypothesize, 
gun shows are associated with 
a short-term, local increase in ho- 
micide, then failing to identify 
20% of gun shows—and as a result 
wrongly assigning homicide data 
for the weeks following those 
missed shows to the no-gun-show 
category—introduces a bias to- 
ward the null (a finding of no 
effect) in their analysis. 

There is significant heterogene- 
ity among gun shows,°"* but 
Duggan et al. treat them equally. 
Smaller shows can involve fewer 
than a dozen gun sellers, all of them 
small-scale operations and most 
of them selling only used guns. 
Such events may have only a few 
dozen attendees on site at any time. 
Some shows are sponsored by gun 
collector organizations and display 
primarily antique weapons with lit- 
tle, if any, utility for criminal pur- 
poses. Large general-purpose gun 
shows involve hundreds of gun 
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vendors, and individual vendors 
may have more than 1000 new 
guns on display.° Any impact of 
these large shows might easily be 
masked by the limited effect of the 
more numerous smaller events. 


CONFLICTING RESULTS 
AND PRIOR RESEARCH 


An earlier version of the study 
by Duggan et al. yielded different 
results.’? In that study, the relevant 
portion of which was limited to gun 
shows in California, census place 
and not zip code was used as the 
geographic unit of analysis and the 
authors argued specifically that zip 
code was a less appropriate choice. 
In the earlier version, there was also 
a statistically significant overall in- 
crease in gun deaths in the 4 weeks 
following gun shows. When deaths 
were disaggregated by type, the 
effects remained significant for sui- 
cide and unintentional death. 
Nothing is said in the new version 
of the study about the prior and 
very different findings or in rebuttal 
to their own prior arguments 
against using zip code as a geo- 
graphic unit of analysis. We do not 
take a position on whether zip code 
or census place is the more appro- 
priate measure, as our concern is 
with the larger issue of geographic 
restriction. The unexplained rever- 
sal and the lack of robustness of the 
study findings are independent 
sources of concern about the val- 
idity of those findings. 

An entire body of literature, 
reviewed elsewhere,”°" has 
documented a positive relationship 
between gun ownership and both 
gun homicide and gun suicide. One 
of these studies, though it is not 
often cited in this context, was 
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conducted by Duggan.”* In that 
study, titled “More Guns, More 
Crime,” Duggan used sales of Guns 
& Ammo magazine as his proxy 
measure for gun ownership. He 
demonstrated the validity of his 
proxy measure in part by demon- 
strating that state-level magazine 
sales were very closely linked to 

a state-level count of gun shows. 
The regression coefficient was 
0.995; a 10% change in either 
measure was associated with a 10% 
change in the other. As a practical 
matter, the study would have found 
a nearly identical relationship 
between gun ownership and gun 
crime had the number of gun 
shows, not the number of maga- 
zines sold, been used as a proxy 
for gun ownership. Such a study 
might have been titled, “More 
Gun Shows, More Crime.” 


CONCLUSIONS 


Taking all these points together, 
we believe the study by Duggan 
et al. is fatally flawed. The specifics 
of the hypothesis tested in the 
study essentially precluded the 
possibility of a positive finding, 
and the statistical power problem 
in itself is sufficient to render the 
study’s results essentially incapa- 
ble of interpretation.?* Beyond 
this, the data used for the analysis 
contain important ascertainment 
and classification errors that intro- 
duce biases toward the null, and 
the results are not robust. 

Given these difficulties, it is 
perhaps not surprising that some 
of the study’s findings have no face 
validity. For example, in some 
models Duggan et al. find signifi- 
cant changes in gun-related deaths 
in the week beginning on the first 
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day of the gun show for shows 
occurring in California. But Cali- 
fornia has a 10-day waiting period 
for all gun sales, including those 
at gun shows; transfers of posses- 
sion cannot legally occur during 
that first week. 

Finally, we note that Duggan 
et al. argued based on their find- 
ings that regulating gun shows 
has no effect. But California’s 
regulations on gun shows pro- 
duce observable effects that are 
likely to benefit public safety, in- 
cluding decreases in anonymous, 
undocumented private-party gun 
sales and illegal surrogate or 
“straw” purchases.”* It has also 
been shown that regulating private 
party gun sales generally, not just at 
gun shows, is strongly associated 
with a decrease in intrastate gun 
trafficking, even when gun owner- 
ship rates and other gun regulations 
are taken into account.”* m 
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